Motor performance deficits of older adults are due to dysfunction at multiple levels. Age-related differences have been documented on executive functions; motor control becomes more reliant on cognitive control mechanisms, including the engagement of the prefrontal cortex (PFC), possibly compensating for age-related sensorimotor declines. Since at functional level the PFC showed the largest age-related differences during discriminative response task, we wonder whether those effects are mainly due to the cognitive difficulty in stimulus discrimination or they could be also detected in a much easier task. In the present study, we measured the association of physical exercise with the PFC activation and response times (RTs) using a simple response task (SRT), in which the participants were asked to respond as quickly as possible by manual key-press to visual stimuli. Simultaneous behavioral (RTs) and electroencephalographic (EEG) recordings were performed on 84 healthy participants aged 19–86 years. The whole sample was divided into three cohorts (young, middle-aged, and older); each cohort was further divided into two equal sub-cohorts (exercise and not-exercise) based on a self-report questionnaire measuring physical exercise. The EEG signal was segmented in epochs starting 1100 prior to stimulus onset and lasting 2 s. Behavioral results showed age effects, indicating a slowing of RTs with increasing age. The EEG results showed a significant interaction between age and exercise on the activities recorded on the PFC. The results indicates that: (a) the brain of older adults needs the PFC engagement also to perform elementary task, such as the SRT, while this activity is not necessary in younger adults, (b) physical exercise could reduce this age-related reliance on extra cognitive control also during the performance of a SRT, and (c) the activity of the PFC is a sensitive index of the benefits of physical exercise on sensorimotor decline.